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I. INTRODUCTION 



Current numerical weather prediction systems contain 
errors from various sources; data inaccuracies, initializa- 
tion, irregular or inadequate distribution of observations, 
or systematic errors from the model itself (Haltiner and 
Williams, 1 980). 

Prediction problems arising from these errors can be 
divided into three categories, depending on the time scale 
of the forecast (Gronaas, 1 982). The first are the short 
range forecasts, mainly affected by the initial conditions 
of the model. The second are the long - term forecasts 
which are largely independent of the initial data. This 
includes the problem of the model drifting from the real 
atmospheric climatology to its own climatology. The third 
category between these extremes is the medium range fore- 
cast. Bengtsson and Simmons (1983) define this time scale 
as being a few days to a week or two. In tnis study the 
medium range forecast category will be defined as four to 
five days in length. 

Medium range forecasting today is in much the same posi- 
tion as short range forecasting was in the in the early days 
of numerical weather prediction (NWP) . Now, as then, one 
finds that the forecasts have some useful information in 
them, but they also contain a considerable amount of system- 
atic error (Gronaas, 1982). 

Compared to the early NWP models, however, the system- 
atic errors in today's complex NWP models are much less 
predictable (Gronaas, 1982). The end result is that the 
meteorologist forecasting in the medium range must use 
subjective synoptic skills to extract useful information 
from the weather forecasts or rely on climatology and 
persistance (usually as a last resort) to sort out the poor 
from tne good forecasts. 
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It is known that there are times when the medium range 
forecasts produced by any of the major operational centers 
are exceptionally accurate, and other times when they are 
exceptionally poor. For example, time series representa- 
tions of the Anomaly Correlation Coefficient (ACC) in 
studies by Gronaas (1982) , Bengtsson and Simmons (1983) and 
Bengtsson (1 985) at 
Forecasts (ECMWF) ha 
the medium range fore 
Assuming that values 
or 0.6 indicate the 
(e.g., Hollingsworth 
are periods of time w 
beyond forecast time. 

These "Spells of Time when the ACC is Relatively High" 
(hereafter referred to as a STARH and pronounced 'star') 
occur without any apparent periodicity and last anywhere 
from around two to twenty-two days long- Although Bengtsson 
and Simmons (1983) did not give a reason for STARH exis- 
tence, they stated that investigating the reason for these 
spells of high and low predictability is an important 
research topic 
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Bettge and Baumhefner (1 984) conclude in their study of 
the FGGE data that model forecast skill was strongly influ- 
enced by the planetary wave structure. The long-term oscil- 
lations in the forecast skill were found to be related to 
rapid transitions in the planetary waves; good predictions 
occurred when the waves were relatively stationary (blocking 
situations) and poor forecasts were found during the tran- 
sition periods. These poor predictions, or "Forecasts of 
Low Predictability" are hereafter referred to as FLOPS. 

This information suggests that it might be possible to 
objectively guantify parameters of the atmospheric flow 
pat tern s that would explain the cause of these high and low 
predictability periods in medium range forecasts. It is 
interesting to note that Elsberry, et al. (1985) indicate 
that there has been no method developed or tested to deter- 
mine the accuracy of a medium range forecast based on the 
day one forecast error. Also, Bengtsson (1985) stated that 
the cause of high and low predictability periods is a funda- 
mental guestion to which no conclusive answer can yet be 
given. 

The objectives of this study, then, are twofold. First, 
to develop a method to objectively choose high (STARH) and 
low (FLOP) predictability periods. Second, to investigate 
the flow patterns of the chosen periods and study the effect 
on the forecast score of the following: 

• Persistence 

• Early error growth 

• Error pattern locations 

• Average height field tendencies and standard deviation 
differences 

• Zonal wavenumber structure 

• Meridional wavenumber structure 

Chapter two of this study discusses the the source of 
the data used in this investigation and the method employed 
to score the forecasts. Chapter three details the objective 
method for choosing STARH and FLOP periods and a summary of 
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the periods selected. Chapter four presents the analysis 
methods employed to investigate the good and bad forecast 
periods and a summary of the number of periods accounted for 
by each analysis method. The conclusions and recommenda- 
tions for future research are summarized in Chapter five. 
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II. DATA SUMMARY 415 SCORING METHOD EMPLOYED 



A. DATA SUMMARY 

Medium range forecasts produced by NOGAPS (Navy 
Operational Global Atmospheric Prediction System) are 
studied in this thesis. The data was retrieved from the 
Model Output Statistics data-base stored at the Fleet 
Numerical Oceanography Center (FNOC) in Monterey, 
California. 

NOGAPS is a nine layer gridpoint model with a horizontal 
resolution of 2.4° latitude by 3.0° longitude. The maximum 
NOGAPS forecast length is five days, generated once a day 
from the 0000 GMT analysis data. This limits the study to 
five-day forecasts. 

The current version of NOGAPS, 2.1, was implemented on 8 
December 1983. Since that time, an error in the latent 
heating term was detected (it was one half the correct 
value). This error was corrected in April 1984. 

Only the northern hemi sphereic section of the global 
data base was investigated. This is because forecast accu- 
racy and usefulness is substantially less in the southern 
hemisphere due to a sparsity of data. This was demonstrated 
to be the case for the ECMWF model by Bengtsson and Simmons 
(1983), and was also found to be true of the NOGAPS model in 
preliminary investigations of this study. 

The winter season forecasts were used for the investiga- 
tion of the high and low predictability periods, since 
winter is the season with the largest number and the most 
energetic systems. Also, it is recognized that model ACC 
scores tend to be higher in winter than in summer. 

Tnere were only two winter seasons of data available 
since the NOGAPS 2.1 model became operational on 8 December 
1983. The total number of days possible to use in this 
study were 83 days from the 1983 winter season (8 December 
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ACC produced is low. For example, a forecast poor in 
synoptic detail or timing may score badly while still giving 
some indication of an overall change in the weather type 
(Bengtsson and Simmons, 1983). 

Following Miyakoda (1972), the ACC is defined to be: 

2 (F-C) *(0-C) 

ACC = (2.1) 

>/<2( F-C) 2 * 2(0-0 2) 

where: F = Forecast parameter 

0 = Observed value 
C = Climatology value 

Since the climatology data base was not available from FNOC, 
a monthly mean 500 mb height was used for climatology. 

A further qualification was made as to the latitudes 
over which the ACC was calculated, following Bengtsson and 
Simmons (1983). Only those grid points within the latitude 
band 20° to 82.5° were included in the calculation to avoid 
the tropics and the polar region. This is done because all 
present day NWP models are poor in the tropics and finite 
differencing type models reguire special handling at the 
poles (Haltiner and Williams, 1980). 



15 



Ill- DISCUSSION OF SELECTED PERIODS 



A. OBJECTIVE DEFINITION OF PREDICTABILITY 

An objective measure of what constitutes a high or low 
predictability period in a numerical weather prediction 
forecast is desired. Hollingsworth et al- ( 1980), states 
that an anomaly correlation coefficient value of 0.5 or 0.6 
is the limit of useful predictability. 

Gronaas (1 982) divides the high versus low predict- 
ability forecasts by comparing the average 3-, 5-, and 7-day 
ACC values for a period of interest (the periods being from 
3 to 26 days long) to the same for the yearly averaged 
values, and he subjectively decides if each one is a high or 
low predictability period. 

This study will only examine the most significant 
periods of the five-day forecasts that were well or poorly 
forecasted.. To this end the high and low predictability 
periods are chosen as follows: 

• The ACC is calculated in the latitude band from 20° to 
82.5° N for the entire winter. 

• The good periods are initially determined by those 
periods where the ACC is above 0.6 (the limit oi useful 
skill) , and the poorly forecasted periods being those 
below 0.4 (to limit the data set to the very worst 
cases) . 

• The ACC for these periods are averaged, and compared 
to the mean and standard deviation of the ACC for the 
entire winter data set. 

• If the average ACC for the period is one or more stan- 
dard deviations above or below the long term mean, the 
period is accepted as either a STARH (nigh score) or a 
FLOP (low score) . 

B. SUMMARY OF SELECTED PERIODS 

Figs. A . 1 and A. 2 show the time series of the anomaly 
correlation coefficient plotted as a function of the day 
(solid line) for both winter data sets. The convention 
adopted for these graphs is that the score listed above a 
day is the ACC score of the five-day forecast that was 
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generated on that day (and whose verification time was 120 n 
later). Also shown on these figures is the persistence ACC 
score, plotted as a function of the day (dashed line). This 
score, computed as was the five-day forecast ACC, uses the 
analysis field as the five-day forecast. 

Table 1 is a summary of the STARH and FLOP periods 
selected. The statistics listed in Table 1 show each period 
meets the selection criterion established in the previous 
section. Shown are the thirteen selected periods, 
consisting of six STARH and seven FLOP periods evenly 
divided between the two winters. The rationale for the 
numbers assigned to the periods is that the one-digit 
numbers are periods from the 1983-84 winter season, while 
the two-digit numbers refer to periods from the 1984-85 
season. Winter 1983-84 periods are not consecutively 
numbered because other periods had been initially selected 
but were later discarded when the selection criterion was 
imposed . 

The period cf time over which the STARHs or FLOPS exist 
varies from the shortest of two days (periods S5, F4 , and 
S12) to the longest of twelve days (period S10). It is also 
interesting to note that like periods do not all occur in 
the same month or m a close period of time, but exist in 
more of a random distribution pattern. 

These selected periods are now studied in detail to 
better understand the reasons for such variation in medium- 
range forecast performance. 
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IV. IN VEST JG ATI QMS ON HEDIOM RANGE FORE CASTIN G 



The flow patterns of the chosen periods are studied in 
order to quantify the forecast score variance due to persis- 
tence, early error growth, error pattern locations, average 
height field tendencies, standard deviation differences, 
zonal wavenumber structure and meridional wavenumber 
structure. 

A. THE ROLE OF PERSISTENCE 

Persistence of the atmospheric flow patterns is natu- 
rally a candidate to explain the existence of high and low 
predict abilit y periods. If persistence is a strong factor, 

one would expect that a flow pattern that exhibited little 
change through the forecast period would verify with a high 
score, while periods of strong flow pattern change would 
verify poorly. 

Figs. A.1 and A. 2 show the persistence score as a func- 
tion or forecast day. Examining these figures for the STASH 
and FLOP periods, one sees that persistence is a poor indi- 
cator of model performance. For example. Fig. A.1 shows 
persistence ACC score to peak at the same time as STASH SI 
(26-30 December 1983), but it also peaks at the same time as 
FLOPS FI (30-31 January 1984 ) and F8 (15-21 February 1984). 

The number of selected periods explained well or poorly 
by the persistence method is listed in Table 2 One finds 
that five periods can be accounted for by the persistence 
method (SI, F4, F7 , S11, and F12) , but six others (S2, S5, 

FI, F8, S12, and F11) are periods where the persistence 

value is just opposite to what would be expected using this 
analysis method. It is evident that with as many periods 
occurring opposite of what is expected in both winters and 
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for both STARHs and FLOP s , persistence is not a good indi- 
cator of model forecast score in the medium-range. 

Additional tests were run on the correlation between the 
model forecast ACC time series curve and the persistence ACC 
curve of Figs. A.1 and A. 2 The results are shown in Table 3 
for winter 1983-84 and in Table 4 for winter 1984-85. If 
persistence was to be a good indicator of the forecast ACC, 
the correlation between the model and persistence forecast 
ACC curves would be high (close to one). Any correlation 
would produce a value at least above zero. It can be seen 
in these tables that only for one month (December 1983) is 
the persistence ACC curve highly correlated with the fore- 
cast ACC curve (correlation value of 0.47) . This correla- 
tion is also evident on Fig. A.1 Three of the months show 
negative correlation values (January and February 1984, and 
February 1985), while the other two months have small posi- 
tive values. 

It is interesting, that in one month (December 1983) of 
the six months studied, the medium-range forecast score 
variance was so highly correlated with persistence. Quiroz 
(1984) states that December 1983 was a month of unusually 
strong blocking patterns over North America. This may, in 
part, explain wny the December ACC score so strongly paral- 
leled the persistance score. 

In any case, the fact that persistence is not a good 
indicator of forecast skill in this data set is consistent 
with the results of Bengtsson and Simmons (1933) who also 
found high medium-range forecast scores in times when a 
large changes occurred in the height fields through the 
forecast period. 



B. THE ROLE OF EARLY ERROR GROWTH 

More data available for a NWP model initialization 
should lead to better short and medium-range forecasts. 
This suggests that one reason for poor model performance in 
the medium-range might be the quality or quantity of the 
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initial data. If model skill is affected by these errors in 
the initial data field, one would expect to see a lower ACC 
score immediately in the forecast period for the FLOP 
periods, and a higher ACC score in this same period for the 
STARHs. 

The average ACC values of each selected period as a 
function of time are presented in Figs. A. 3, A. 4, A. 5, and 
A. 6 . These figures show the trend of the average forecast 
score for each period over the~ five-day forecast period. 
Comparing the STARH periods of winter 83-84 on Fig. A. 3 to 
the FLOPS of that winter on Fig. A. 4, little difference 
between the STARHs and FLOPs is present out to 48 hours. 
Only one FLOP, F8, is seen to have a noticeably lower ACC at 
the 48- hour pcint. F4 and F7, however, cannot be distin- 
guished from the STARHs at this time, and FI is only 
slightly lower than the the lowest scoring STARH at 48 hours 
<S2) . 

It is interesting to note that F8, although scoring much 
lower than the other FLOPs at 24, 43 and 72 nours, is at 

almost the same ACC value at 96 and 120 hours. This indi- 

cates that early errors for F8 did not produce a 120-hour 
ACC score significantly below the other FLOPs. 

Comparing the STARHs and FLOPs from winter 84-85 on 

Figs. A. 5 and A. 6 shows similar results. At 48 hours the 

ACC of the STARHs are all higher than the FLOPs, but the 

STARHs and FLCPs are very close on the ACC scale. They are 

all within 0.05 of each other, and the highest FLOP is only 
about 0.02 below the lowest STARH. Although there are not 
enough data to assign a statistical significance to this 
grouping of points, it is clear that for both winters the 
largest difference in ACC between the STARH and FLOP periods 
lies at the 72-hour point and beyond. In the summary of 
results (Table 2) only one of the 13 periods (F8) shows an 
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THE ROLE OF ERROR PATTERNS 



1 . 



S umm ary 
Per 1073s 



of Error Sources for the STARH and FLOP 



This section summarizes the features in the analysis 
charts of the 500 mb height field that differed from the 
forecasted height field for all the good and poor forecast 
periods. By comparing the error patterns (generated by the 
subtracting the 500 mb verifying day analysis from the 500 
mb five-day forecast) to the verifying day analysis, one 
could identify the flow pattern features related to the 
error patterns. Retracing in time the evolution of the 
feature, it was possible to gain some insight as to why some 
periods scored poorly and why others scored rather well. 

Examining all the selected periods in the winter 
season 1983-84 in this fashion revealed that the major 
sources of errors in the forecast field could be reduced to 
a few features in common to all the periods. While tne 
basic cause of the errors was found to be the same for both 
the STARHs and FLOPS, it should understood that the sizes 
and amplitudes of systems causing the error patterns in the 
STARHs were much smaller. The most common features that 
caused the large error patterns are listed below: 

• Formation of a major ridge late in the forecast period 
(around day three to day five) . 

• The formation of a major trough late in the forecast 
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period (around day three to day five). 

• The rapid deepening of a low pressure center or devel- 
opment of a shortwave trough within days three to five 
of the forecast period. 

• The formation of a cutoff low pressure center after dav 
two of the forecast period. 

• The regression of a low or high pressure center after 
day two of the forecast period. 

• Inability to maintain the intensity of major high or 
low pressure centers that persist throughout the fore- 
cast period. 

These features were almost always under forecasted 
by NOGAPS. That is, the highs were too low and the lows 
were too high in the five-day forecast. There were 
instances in which the sense of the error was just the oppo- 
site as this, but these cases were by far fewer and 
accounted for a only a small part of the total error field. 

It is interesting to note that in the summary of the 
errors listed previously, most of the error causing systems 
were ones that developed after day two in the forecast 
period. This corresponds with Figs. A. 3 through A. 6 which 
show the largest difference in ACC scores between the STARH 
and FLOP periods is after day two as well. 

2. Geog rap hic Location of Errors 

An example of the types of errors encountered is 
shown on Figs. A. 7 through A. 11 . Fig. A. 7 is a northern 
hemisphere polar stereographic (PS) 500 mb height analysis 
from 23 December 1983. The five-day forecast made from this 
analysis was a FLOP (F4) and verified against the 500 mb 
height field analysis of 28 December 1983, shown in Fig. A. 9 
The forecast errors (forecast minus observed height values) 
are shown on Fig. A. 8, with contour intervals of 160 meters. 
Comparing this with Fig. A. 9 one can quickly discern that 
the largest area and amplitude errors are associated with 
features that intensified or developed within the forecast 
period. The most noticeable features on Fig. A. 9 associated 
with large error patterns are the omega block over western 
Europe, the family of lows across the North Pacific Ocean, 
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the ridge along the west coast of North America, and the 
trough over north central Canada (the area around Baffin 
Island) . 

The five-day forecast generated from the 28 December 
analysis was a STARH, and verified against the analysis of 
02 January 1984, shown on Fig. A. 11 The plot of the error 
patterns for this forecast, similar to Fig. A. 8, is shown on 
Fig. A. 10 Not surprisingly , the size of and number of the 
error patterns greater that 160 meters is much smaller, 
indicating the forecast field captured most of the changes 
that had occurred. 

The features on Fig. A. 11 that correspond to the 
error patterns of Fig. A. 10 are not nearly as spectacular as 
they were for Fig. A. 8 Two of the error patterns (over 
England and in the mid-Pacific Ocean) are associated with 
mostly zonal flow, while two others (Greenland- Newf oundland 
area and the Black Sea region) are associated with short 
wave troughs. The remaining error pattern is related to a 
diffluent trough along the west coast of North America. 

In looking at all the error pattern maps for all the 
STARH and FLOP periods of the 83-84 winter, it was evident 
that most of the differences in error patterns between the 
STARH and FLCP periods are within the latitude band of 
roughly 30° to 60°. Both STARH and FLOP periods were seen 
to have large errors north of 60° north latitude, but not 
too surprisingly, only the FLOP periods had large area and 
amplitude errors in several locations within the 30° to b0° 
latitude band. 

Both STARHs and FLOPS had errors caused mostly by 
systems that developed late in the forecast period (after 
day two). The FLOP periods had large scale development of 
systems roughly within this latitude band over the forecast 
period, while the STARHs did not show this feature. To 
determine if these developing large amplitude error patterns 
were occurring systematically in the same geographic loca- 
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verifying day of 02 Janaary 1984 (Fig- A- 11) and the error 
pattern associated with the five-day forecast (Fig. A- 10) 
presents an excellent example of this difference- The large 
blocking systems of 28 December decayed and were minimal 
sources of error while no new systems developed. 

The tendency of the STARH periods to exhibit less devel- 
opment and more of a decay of the existing high amplitude 
systems can be described as the flow shifting from a low to 
a high index regime (meridional flow to zonal flow tran- 
sition) - The opposite would then apply to the FLOPS. 

NOGAPS systematic errors at 500 mb (Boyle and Wash, 
1985) are to fill troughs and weaken ridges. Thus, NOGAPS 
will verify better in the medium-range when the atmosphere 
trend (relaxing of high amplitude systems) follows the model 
systematic error trend. 

To objectively quantify this flow pattern behavior, the 
500 mb analysis height fields were averaged between 30° and 
60° N latitude (the latitude band in which it appeared the 
STARHs had less errors and the FLOPS had more errors). Each 
day's average values were plotted as a function of longitude 
in a Hovmiiller diagram as shown in Figs. A. 14 and A. 15 . 
The zonal mean is removed so the low height areas show 
dashed lines, the heavy line is the zero contour, and the 
high height areas show as solid lines. The heights were 
contoured at an interval of 80 meters. 

These diagrams show the height field anomaly at each 
longitude for each day. Time increases towards the top of 
the graph, which allows one to discern how the anomalies 
(the highs and lows) change over time with respect to any 
one. location. The figures show the typical features of the 
winter 500 mb patterns, the long wave troughs at longitudes 
150°E and 70°W, with ridges at 70°E, 130°W and 20°W. 
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If the proposed mechanisms are correct, one would expect 
that as you move forward in time in a STARH period from the 
day a forecast was generated to its verifying time, the 
highs would be decaying and the lows would be filling- It 
is pleasing to note that this is exactly what is seen for 
some of the periods. In Fig- A- 14 , for winter 1983-84, 

this is the case for periods Si and S 5- By examining the 
height field for a day in these periods, and then glancing 
forward in time (towards the top of the figure) , one can see 
that over the five-day forecast period, the lows are filling 
and the highs are decaying for the most part. The same is 
true of periods S10 (the last. half), and S12 on Fig- A- 15 - 
The shifts in height anomalies expected for the FLOP periods 
are just the opposite as those for a STARH (i.e., the lows 
should deepen and the highs should build over the forecast 
period) . This can be seen for F4 and F7 in Fig- A- 1 4 and 
for F10, F11 and F12 on Fig. A. 15 

Unfortunately, the height anomaly trend for each type of 
period does not always occur as expected- The first part of 
S 1 0 on Fig. A. 15 is the best counter example. The highs are 
building and the lows are deepening, but the model scored 
high using the ACC (this is certainly what the numerical 
modelers like to see). These are periods when the model can 
successfully predict wave amplification in the medium-range. 
Also, using shifts in height anomalies over time as a method 
of determining model performance for the medium-range fore- 
casts does net uniquely define STARH and FLOP periods. 
There are several periods that show the tendencies for a 
STARH (FLOP) period, yet did not score significantly high 
(low) with the ACC. For example, on Fig- A- 14 one suen 

STARH-like period is from about 8 to 10 February 1984, and a 
FLOP- like period is around 2 to 3 January 1984. This means 
that although the technique works for many of the STARHs and 
FLOPs, it is not unigue solely to those periods. 
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To better illustrate the shifts in the height anomalies 
over time, the standard deviation of the mean height is 
calculated for the height field represented on Figs. A- 14 
and A. 1 5 . The standard deviation of each day’s height 

field is shown on Fig. A. 16 for winter 83-84 and Fig. A. 17 
for winter 8 4-8 5. 

It is expected that a low index period would have a 
large standard deviation, as compared to a high index period 
where a smaller standard deviation value would be calcu- 
lated. Detecting the change in the standard deviation value 
over the five-day forecast period is simplified by using 
Figs. A. 18 and A. 19 (standard deviation difference graphs 
for 83-84 and 84-85). This graph is constructed by 
subtracting the standard deviation of the day at the veri- 
fying time from the value at the analysis time. The differ- 
ence is plotted as a function of time when the five-day 
forecast was generated. 

It is expected that the STARHs would be indicated as 
positive peaks (showing a tendency of moving from a low to a 
high index regime) and FLOPS as negative minima. This anal- 
ysis technigue has some successes and some interesting 
results (results occurring opposite to expectations are 
termed ••interesting 1 ' vice ••bad" or "failures"). For 
example, in Fig. A. 19 (winter 1984-85), the two largest 
positive peaks precisely define two STARH periods (S 1 2 ani 
S10, as indicated). Analogously, F10 (a FLOP) is precisely 
defined as a large "valley" in the negative portion of the 
graph as expected. This indicates that the shift in height 
anomalies detected on Figs. A. 14 and A. 15 can be objec- 
tively guantified and is successful in describing some 
periods . 

The other three periods of this winter, however, are not 
guite as successful. The standard deviation difference for 
STARH ( Si 1 ) is generally positive, but not exceptional (the 
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period 21-25 December which did not have a spectacularly 
high or low ACC was generally more positive than the period 
27-31 December of S11). Similarly, the standard deviation 
differences of periods F11 and F12 are both mostly negative 
(as expected), but they are not within distinctly low 
negative areas (the period 9-12 February is a distinct 
minimum, but is not a STARH or a FLOP according to the ACC 
scores) . 

The most interesting part of Fig. A. 19 is the first part 
of S10 that lies in a large negative portion of the graph 
(22-29 January) . This region has every indication of being 
a poorly predicted period, yet the model forecasted the 
changes that occurred very well. 

The successes are fewer for the 1983-34 winter period 
(Fig. A. 18). For example, S5 and SI are within positive 
peaks of the graph, but other areas of higher positive 
values exist (15-16 December, 6-10 January, and 17-18 
January) that were not spectacular ly high or low ACC 

periods. Also, the FLOPS FI, F4, and F7 do not lie exactly 

in a local minimum portion of the graph. Nevertheless, 
these FLOPS are almost all on the negative side of the 
graph. Neither does SIARH S2 lie in a peak of the standard 
deviation values, and the last day of the period (21 January 
1984) is in the negative portion of the graph. 

The two most interesting features of this winter are 
that the large minimum at 22-23 January does not delineate a 
FLOP, and that F8 has standard deviation differences 
entirely on the positive side of the graph. This means that 
the model did poorly even though the flow regime was 
changing to a more zonal flow pattern (expected to produce 
good forecasts) . 

For both winters, these graphs also serve to show 
periods when the ACC was neither s pectacularly high or low, 
yet look like STARHs or FLOPS because of the large positive 
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or negative standard deviation differences- This more 
clearly illustrates that this height anomaly tendency is not 
a unique indicator of a STARH or FLOP period. 

The summary of which periods were not described well by 
this method is shown on Table 2 In all, four periods had 
expected results (Si, S5, S12, and F 10), one had results 
opposite to expectations ( F8) , and the remaining eight 
periods had results that were generally as expected, but 
were either not spectacular or had one or more forecasts in 
the period that did not fit the hypothesis- 

E- ZONAL WAVENUMBER STRUCTURE 

Bettge and Baumhefner (1984) used zonal wavenumbers one 
and two in describing the systematic errors in NMC (National 
Meteorological Center) forecasts. They conclude that the 
power (amplitude) distribution in the wavenumbers has a 
bearing on the predictability of a forecast, such that when 
the long waves were stationary the medium-range forecast was 
good. However, when the long waves were in transition, the 
medium- range forecast was poor. 

This technigue can also indicate the flow characteris- 
tics of a height field. Meridional flow is related to high 
power (amplitude) in the lower wavenumbers (waves 1-3), 
while zonal flow is more a function of the power in the 
higher wavenumbers (4-6) . 

To investigate how the zonal wavenumber structure of the 
height fields is related to medium-range predictability, the 
Fourier decomposition method is used to determine the waven- 
umber versus amplitude spectrum of each 500 mb analysis 
height field- The standard fast Fourier transform method 
was employed (IHSL, 1982). The 500 mb analysis field height 
anomalies in Figs. A. 14 and A. 15 were used as the input 
waveform for the Fourier dec omposit ion routine. The ampli- 
tudes (in meters) for each day are plotted as a function of 
the wavenumber in Figs. A. 20 (winter 83-84) and A. 21 (winter 
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time is 



84-85) - As in the previous Hovmiiller diagrams, 
increasing towards the top. 

No one "signature" of a denoted STARH or FLOP period is 
common to both winters in this harmonic analysis. For the 

1983- 84 winter season, the most characteristic feature of 
the STARHs is the peak in wave number three (often associ- 
ated with a blocking situation as in Fig- A. 9) and rela- 
tively lower amplitudes in waves two and four. However, 
this characterization of the wavenumber structure of STARHs 
and FLOPS in the 83-84 winter is not unigue to these periods 
alone. For example. Fig. A. 20 shows that there are peaks in 
wave three on 7 and 17 January 1984 and on 9 February, yet 
these are not STARHs. Similarly, 20 December 1983 and 14 
February have distinct minimums in wave three, yet lie 
outside of the FLOP periods. 

There is no correspondingly high amplitude in wave 
number three for all the STARH periods of winter 84-85 (Fig. 
A. 21). No conclusion can be drawn about a characteristic 
structure that would indicate a STARH or a FLOP period since 
the data set is too small considering the variances encoun- 
tered within the wavenumber amplitudes. 

The indication, discussed earlier, of STARHs to exist 
when the flow pattern shifts from a meridional to a zonal 
pattern can also be detected as a shift in the amplitude of 
the zonal wavenumbers from the analysis amplitude spectrum 
to that of the verifying amplitude spectrum. 

To facilitate detecting these shifts in the amplitude of 
the major wavenumbers for both winters, a Hovmiiller diagram 
has been constructed that shows the change in amplitude over 
the five-day forecast period for each wavenumber. Fig. 
A. 22 graphs the five-day tendency of the wavenumbers in 
winter 1983-84, and Fig. A. 23 is the same for winter 

1984- 85. The tendencies are calculated by subtracting, for 

each wavenumber, the day one value from the day five value. 
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Positive values reflect growth in a wavenumber, and negative 
values reflect decay. The convention for these diagrams is 
that the difference is plotted as a function of day one and 
(as before) time increases as one moves toward the top of 
the diagram. 

It is expected that a STARH period will experience a 
decrease in amplitude in the longer waves and possibly an 
increase in power in the higher wavenumbers. A shift in 

amplitude in this sense would be an indication that the flow 
pattern is shifting from a 1 ow index (meridional flow) to a 
high index (zonal flow) regime. FLOPS would be expected to 
reflect just the opposite behavior. 

This analysis technique meets with more success than did 
the standard deviation differences technique, as more 
periods are explained by this method. Table 2 lists seven 
of 13 periods that have shifts in wavenumber amplitudes as 
expected (S2, FI, F4, S 1 1 , S12, F11 and F12), while the 

standard deviation method can only account for four of the 
13 periods. On the other hand, period F8 shows a tendency 
that is just the opposite to what is expected. This period 
also showed a similar result for the standard deviation 
method. The remaining five periods show show some of the 
expected shifts in zonal wavenumber amplitude, but also some 
complex behavior not anticipated. For example, in Fig. A. 22, 
STARH SI does show a decrease in amplitude at the beginning 
of the period (26 December) in wave two, but wave one 

increases though most of the period. On 27 December wave 
three shows decay, and wave five growth (as expected) , but a 
shift occurs on 29 December when wave one begins to decay 
and wave two shows growth. Waves four and five (as 
expected) show growth from then until the end of the period. 

Other periods like this are S5 and F7. In the second 
winter (Fig. A. 23) the periods showing expected shifts in 
wavenumber amplitudes are S11, S12, and F 1 1 - Those showing 
more complex behavior are SI 0, F10 and F12. 
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Unfortunately, the data set for these two winters is too 
small to be able to statistically show that one pattern of 
behavior in the complex cases is significant, but it is 
clear that many periods are explained very well by this 
technique. Most interesting is period F8 which (as it did 
in the standard deviation difference results) shows results 
just the opposite to what was expected (i.e., the long waves 
gained power, while wave four and six lost power over the 
forecast period). Also intriguing are those periods not 
scored hign or low with the ACC, yet exhibit the same 
tendencies as a STARH or a FLOP (such as 15-17 December 
1984; FLOP-like shift in the wavenumbers, and 15-17 January 
1984: STARH-like wavenumber shift). This means that the 

results so far, though promising, are still not unique to 
the STARHs and FLOPS. 

F. MERIDIONAL WAVENUMBER STRUCTURE 

The meridional structure of the flow patterns may be an 
important factor in determining model performance- 
investigation of the north-south extent of the various zonal 
wavenumbers is the subject of this final section. 

It is possible to depict the meridional structure of the 
flow pattern of an analysis field by examining the meri- 
dional wavenumber spectrum. This spectrum can be calculated 
by using a spherical harmonics decomposition of the 500 mb 
height field. 

The spherical harmonics decomposition of the wave field 
depicts the amplitude of wavenumbers in two dimensions, 
zonal and meridional. In this study the spherical harmonics 
decomposition was conducted using a triangular truncation at 
N=33 ("N" being the zonal wavenumber) (Haltiner and 

Williams, 19 80) . 

The wave numbers in the meridional direction are defined 
in this study as "pole to pole" wavenumbers. In plotting, 
the amplitude values (in meters) for the pole to pole waves 
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with the same number of nodes (the same |M-N| wavenumber 
values, where M is the zonal wave number), were summed and 
plotted as a function of pole to pole wavenumber. 

Several plots of the pole to pole wavenumber structure 
did not produce any significant results- It was noted, 
however, in studying the zonal wavenumber structure of the 
1983-84 winter season (Fig. A. 20) that one difference 
between the STARH and FLOP periods was a peak in zonal wave 
three in the STARHs, and a minimum amplitude value in zonal 
wave three for the FLOPS. To examine the meridional extent 
of this wave, the analysis height fields of all the days 
that compose each individual STARH and FLOP period were 
averaged together to form a representative height field of 
that period. This height field was used as the input height 
field for the spherical harmonics decomposition- Fig. A. 24 
shows the meridional wavenumber structure for zonal wave 
three of all the STARH periods of winter 83-84 and Fig. 
A. 25 shows the same for the FLOPS in the 1983-84 winter 
season. 

To interpret these diagrams one must recall that a 
higher amplitude in the lower pole to pole wavenumbers 
( | M — N | nodes) would indicate less meridional structure, or a 
more meridional flow component. If the premise is correct 
that good five-day forecasts are generated on days when the 
flow patterns are more meridional (low index situation) and 
the poor forecasts are generated on days when the flow 
patterns are more zonal (high index situation) , then it is 
expected that the STARHs would snow more amplitude in the 
lower pole to pole wavenumbers, while the FLOPS would be 
expected to show more amplitude in the higher pole to pole 
wavenumbers. 

In contrasting the amplitude in pole to pole wavenumbers 
one and four between Figs. A. 24 and A. 25, the STARH periods 
have a higher amplitude (more power) in wave one than the 
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FLOPS. Also, the FLOPS show more amplitude (power) in wave 
four that do the STARHs. A close inspection of Fig. A. 25 
shows F8 is an extreme case of having more power in the 
higher pole to pole wavenumbers- Recall that F8 provided 
unusual results using the other techniques previously 
discussed in this chapter. 

In summary, all of the STARH and FLOP periods were 
accounted for in the winter season 1983-84 by this tech- 
nique. Table 2 shows this result for the STARHs and FLOPs 
of winter 1983-84 is guite dramatic. This technique works 
well in delineating the good from the poor medium-range 
forecasts in this winter. 

This same approach, however, does not work well for the 
STARHs and FLOPs in winter 1984-85. Figs. A. 26 and A. 27 
illustrate the meridional wavenumber structure of zonal wave 
three for the STARHs and FLOPs of winter 1984-85. The 
construction and interpretation of these figures is the same 
as for those of the previous winter (Figs. A. 24 and A. 25). 
By contrasting the STARHs on Fig. A. 26 to the FLOPs on Fig. 
A. 27, one can detect a slightly lower amplitude in pole to 
pole wavenumbers three and four of the STARHs as compared to 
the FLOPs. This is certainly expected, but overall, there 
is not the strong difference between the periods as in the 
previous winter. F11 and S11, for example, are not very 
different from each other, and the peak in wave two of S12 
is more suggestive of what is expected from a FLOP period. 

The most interesting feature of Fig. A. 26 is the period 
S 10 . The exceptionally large peak in wavenumber one is 
typical of the other STARHs. It should be noted that for 
this analysis technique, only the last half of the days 
composing S10 were included in the averaging process that 
produced the average height field to represent S10 (i.e., 
days 26-31 December were left out). This was done since S10 
was a very long period (12 days long) and since it has 
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already been seen that some other process was occurring at 
the beginning of this period to cause the ACC score to be 
large. By isolating the first half of the period it makes 
more apparent what is happening in the last half of the 
period S10 to cause the high ACC score. Computation of the 
meridional wavenumber structure of the entire period, 
however, showed the graph of S10 to resemble those of S11 
and S12. 

In general, though, the meridional structure of zonal 
wave three is not as successful in the winter 1984-85 
season. It is apparent from the wavenumber structure of tne 
winter season 1 984-85 on Fig. A. 21 and for winter 1983-84 on 
Fig. A. 20 that these two winters definitely had a different 
character. This is emphasised by the strong wave three 
structure of the STARHs in 1983-84, in agreement with Quiroz 
( 1984) on the description of the 19 83-84 winter season. 

Quiroz noted tnat the 1983-84 winter was a. season of 
strong blocking and of intensely cold surface temperatures 
for North America (setting several low temperature records 
and qualifying as one of the six coldest winters on record). 
A quick check of records 'of the 1984-85 winter reveals that 
it was not nearly so spectacular a season as the previous 
winter. 

Since these two winters show a marked difference in 
character, other combinations of wavenumbers were examined 
for the winter season 1984-85 that might show a difference 
in character between the STARHs and FLOPS. Extensive exper- 
imenting and examining the zonal wavenumber structure of the 
1984-85 winter season indicated a difference in the STARHs 
and FLOPS in the zonal wavenumbers four through six. 

Figs. A. 28 and A. 29 show the meridional wave structure 
of zonal waves four through six for the STARH and FLOP 
periods of the 1984-85 winter season. These graphs differ 
from Figs. A. 24 and A. 25 only in that the meridional struc- 
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ture of the combination of zonal waves four through six is 
depict e d. 

These two figures (A. 28 and A. 29) show a clear differ- 
ence in the shape of the curves for the STARHs and the 
FLOPS- As expected, the STARHs have a high peak at pole to 
pole wavenumber one. In general, the FLOPS have a high wave 
one amplitude as well, but .at wave four they clearly have a 
larger amplitude than do the STARHs. F10 does not have as 
high an amplitude at wave four ' as do the other two FLOPs, 
but it also has a correspondingly lower wave one amplitude. 

As seen in Table 2, this technigue correctly indicates 
the expected structure of all the STARHs and FLOPs in the 
winter season 1984-85. However, examining the structure of 
the meridional wavenumbers for zonal waves four through six 
does not provide any differentiation between the STARHs and 
FLOPs of the previous winters. 

G. SUMMARY OF ANALYSIS TECHNIQUES 

Table 2 lists for all the periods selected in both 
winters the ability of the various analysis techniques used 
in this chapter to describe the features expected to be 
found in STARHs and FLOPs. A rather simplistic grading 
scheme is employed to summarize the results of all the tech- 
niques. An "X" listed in the row next to a period indicates 
that the analysis technigue listed in the column above the 
"X" showed expected results. A "/" (slash) denotes some of 
the days of the period showed expected results or that some 
of what was expected to be present was present, but not in a 
spectacular or a unique fashion. A (dash) indicates no 

clear indication was observed, and an "0" is used when 
results were obtained that were the opposite of what was 
expected. 

If one analysis technique was to be a perfect indicator 
of STARHs and FLOPs, one would expect to find nothing but 
"X 1 s" in the column beneath it. No one technique is perfect 
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for the STARHs and FLOPS of both winters, but it is inter- 
esting to note that between both the standard deviation 
differences and the Fourier decomposition differences 
methods only three periods are not fully "as expected". 
Also interesting is that the Spherical Harmonics method can 
explain all the periods of one winter if the correct zonal 
wave or waves are used. Persistence, as discussed earlier, 
is not a consistent indicator of STARHs and FLOPs- This can 
be seen as almost an egual mix of "X*s" and "0's" appear 
under that column. Early error growth is also discounted as 
a means to detect model performance as under the "Early 
Errors" column dash symbols (indicating no clear indication 
observed) appear for all periods except F3. This indicates 
that only in rare occasions does this method apply. 
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V. CONCLUSIONS AND RECOMMENDATI ONS FOR FUTU RE RESEARCH 



The main conclusion of this study is that it is possible 
to quantify certain aspects of flow patterns that are indi- 
cative of STARH and FLOP periods- These indications 
include : 



• Long wave amplitude decay over the forecast period can 
lead to a good forecast, ' while long wave amplitude 

G rowth over the forecast period tends to produce poor 
o recasts- 









Transition of flow pattern from a meridional (low 
index) to zonal (hign index) regime leads to a good 
forecast period. The opposite case leads to a poor 
forecast period. 



STARHs are related to periods 
structure (more power in the 
numbers) while FLOPS are related 
meridional structure. 



with less meridional 
low meridional wave 
to periods with more 



None of these indications, however, show a unique or over- 
whelming result. Periods do exist for each of these indica- 



tors that fit the criterion, but do not score overwhelmingly 
high or low using the ACC. Also, some periods that score 



exceptionally high or low have flow indications (as listed 
above) that are just opposite to what was expected (i.e., a 
STARH had flow characteristics indicative of a FLOP and vice 



versa) . 

There is more confidence placed in the conclusions that 
the variance in the ACC was not due to the following: 

• Persistence as a good indicator of model performance. 

• Early error growth in FLOP periods due to low data 
availability or some critical data missing. 

• Errors consistently occurring in the same geographical 
area causing FLOPS. 

Additionally, the data base available for this study was 
inadequate to provide a statistically significant basis for 
determining confidence levels of the listed indications. 

Based on the findings of this study, several recommended 
future research efforts are delineated here that will hope- 
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fully further our understanding of high and low predict- 
ability periods in medium range forecasts. 

First, it is recommended to investigate the STARH and 
FLOP periods in more detail to determine why certain anal- 
ysis methods used in this study were successful in 
explaining the existence of some high and low predictability 
periods but not for others. 

A second recommendation is to continue computing ACCs of 
NOGAPS 5 day forecasts to build up a more significant data 
base for statistical verification purposes. Expanding the 
seasons examined will also add to the statistical data base, 
but will also indicate wether or not the mechanisms that 
cause STARHs and FLOPS in these other seasons are the same 
as the ones fcr the winter seasons. 

Third, it is recommended to calculate the ACC for the 
five-day forecasts at additional levels in the atmosphere to 
allow other flow patterns to be investigated. The final 
recommendation is to investigate the possibility 
combination of the parameters measured in these 
techniques might provide an indication of the 
predictability of a medium range forecast. 



that some 
analysis 
potential 
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ACC VS DAY 

5 DAY FORECAST AND PERSISTANCE 

WINTER B3-84 
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ACC VS DAY 

5 DAY FORECAST AND PERSISTANCE 
WINTER 84-85 
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Fig- A. 3 ACC versus forecast day for winter 1983-84 
STARH periods- Each ACC value plotted represents an 
average value for the period- 
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Fig- A- 4 ACC versus forecast day for winter 1983-84 
FLOP periods- Each ACC value plotted represents an 
average value for the period. 
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ACC 




Fig- A, 5 ACC versus forecast day for winter 1984-85 
STARH periods. Each ACC value plotted represents an 
average value for the period. 
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Fig. A. 6 ACC versus forecast day for winter 1984-85 
FLOP periods- Each ACC value plotted represents an 
average value for the period. 
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Fig. A. 7 23 December 1983 500 mb height analysis. 

Northern Hemisphere Polar Stereographic projection. 
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Fig- km 8 Error patterns for 23 December 1983 500 mb 
five-day forecast field minus 28 December 1983 
500 mb analysis field. 
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Fig. A. 9 28 December 1983 500 mb height analysis. 

Northern Hemisphere Polar Stereogr aphic projection. 
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Fig- 1.10 Error patterns for 28 December 1983 500 mb 
five-day forecast field minus 02 January 1984 
500 mb analysis field- 
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JAN 1984 
500MB TAUO 
02 JAN 84 
CONTOUR = 6.000*1 
WAVE GROUP = 0 



Fig. A. 1 1 02 January 1984 500 mb height analysis. 

Northern Hemisphere Polar Stereogr aphic projection. 
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g- A. 12 Winter 1983-84 day-five error patterns 
age five-day forecast 500 mb heights minus average 
verifying day an heights). 
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LONGITUDE 

Fig. A. 13 Winter 1984-85 day-five error patterns 
(average five-day forecast 500 mb heights minus average 

verifying day an heights). 
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Fig. A. 16 Winter 1983-84 Standard deviation of average 
daily 500 ab analysis heights of Figure A- 14. 
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Fig. A. 17 Winter 1984-85 Standard deviation of average 
daily 500 ab analysis heights of Figure A. 15. 
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Fig- A- 18 Winter 1983-84 five day standard deviation 
tendencies. Day-1 minus day-5 difference value 
plotted as a function of day one. 
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Fig- A. 19 Hinter 1984-85 five day standard deviation 
tendencies- Day- 1 minus day-5 difference value 
plotted as a function of day one- 
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g. A. 20 Winter 1983-84. Amplitude of zonal 
lumbers of average daily 500 mb height fields from 

Figure A. 14. 
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. A. 21 Winter 1984—85. Amplitude of zonal 
mbers of average daily 500 mb height fields from 

Figure A. 15. 
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Fig. A. 23 Winter 1984-85. Five-day amplitude tendencies 
of zonal wavenumbers. Day-5 minus day- 1 differences 
plotted as a function of day one. 
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AMPL. OF SUMMED MERIDIONAL WAVE NUMBERS 

SPHERICAL HARMONICS: ZONAL WAVE 3 
STARH DATA SETS 
WINTER 03-04 
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Fig. A. 24 iinter 1983-84. Pole to pole wavenuiDer 
amplitude of zonal wave 3 for STARH periQds. 
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AMPL. OF SUMMED MERIDIONAL WAVE NUMBERS 

SPHERICAL HARMONICS: ZONAL WAVE 3 
STAKH DATA SETS 
WINTER 64-65 
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Fig- A-26 Winter 1984-85o Pole to pole vavenuiber 
amplitude of zonal wave 3 for STARH periods- 
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AMPL OF SUMMED MERIDIONAL WAVE NUMBERS 

SPHERICAL HARMONICS: ZONAL WAVES 4-6 
STAR II DATA SETS 
WINTER 64-05 
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Fig- A. 28 Winter 1984-85« Pole to pole wavenumber 
amplitude of summed zonal waves 4, 5 and 6 for STABH periods. 
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Fig- A- 29 Winter 1984-85. Pole to pole wavenuaber 
amplitude of sained zonal waves 4 r 5 and 6 for FLOP periods 



APP EHDIX B 
TABLES 



TABLE 1 

SCJHfiAR Y OF SELECTED PERIODS 



Wi nter 




Period 


Days when 5 
day forecasts 
were generated 


Average ACC 
of period 


83-84 




SI 


26-30 


Dec 


83 


0.705 






S2 


19-21 


Jan 


84 


0. 699 






S5 


08-09 


Dec 


83 


0.642 


Avg.= 
S. D.= 


0.50 0 
0. 134 


FI 


30-31 


Jan 


84 


0.301 






F4 


22-23 


Dec 


83 


0. 295 






F7 


04-06 


Feb 


84 


0.250 






F8 


16-21 


Feb 


84 


0.267 


84-85 




S10 


27 Jan-07 
1985 


Feb 


0. 713 






S1 1 


27-31 


Dec 


84 


0. 675 






SI 2 


14-15 


Ja n 


85 


0.678 


Avg.= 
S. D. = 


0.49 0 

0. 171 


FI 0 


0 8- 12 


Dec 


84 


0. 160 






FI 1 


1 5-20 


Feb 


85 


0.281 






FI 2 


02-04 


Jan 


85 


0. 288 
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TABLE 2 

SUMMARY OF ANALYSIS RESULTS 



winter! PERIOD / ^ / 
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✓ o , 

/ S ' 

/ «® / A 

/ / O' 

/ O’ / 



<0 



/ 6 ©° ' 

b ' O' /s 
/ <0> S © V / * 
/ ^ ^ 4 0 
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® © 

O / 

o © / 

'W.O 



SPHERICAL 

HARMONICS /' 
^ / 



/ W o°/ v ** o N '/ VV* * /£°*Vw ' 



















8 3 — 84 


S 1 
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- 




S 2 
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- 




S 5 
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- 




F 1 
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X 






F 4 
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- 




FT 


X 
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/ 


/ 
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- 




FS 
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■ 1 1 
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F 1 2 
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LEGEND 

’ X ’= Expected result 

* / *= Marginal result 

* - ’ = No significant result 

’ 0 ’= Results Opposite to expectation 
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TABLE 3 

WINTER 83—84 ACC STATISTICAL SUMMARY 



Pa rame ter 


Winter i 

Sum mary 


Dec. 


Jan. 


Feb. 


Average Forecast ACC 


0.500 


1 I 
1 • 1 
1 U1 1 
1 Co 1 
1 1 
1 1 


.53 


.44 


Standard Deviation 


0. 132 


. 1 3 


.13 


.15 


Correlation of 
fcst. and per. 


0. 155 


+ . 47 


1 

1 1 
1 • 

u> 
1 o 
1 


-.20 


Number data points 


87 


3 1 


31 , 


29 



TABLE 4 

WINTER 84-85 ACC STATISTICAL SUMMARY 



Parameter 


Winter 

Summary 


Dec. 


Jan. 


Feb. 


Average Forecast ACC 


0.490 


.46 


.50 


.52 


Standard Deviation 


0.171 


. 1 8 


.16 


. 17 


Correlation of 
fcst. and per. 


0. 138 


+ .26 


+ .24 


-. 11 ■ 


Number data points 


82 


3 1 


31 


23 
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